This invention relates generally to hardware simulation models, and more particularly to providing failure analysis implementing automated comparison of multiple reference models.
Complex computer systems often rely on firmware to implement a portion of the system's design requirements. For example, computer systems with a complex instruction set typically employ a combination of hardware and firmware to implement the instruction set. Instructions which are too complex or too costly to implement in hardware may be implemented in firmware. In addition, firmware can be used to implement many complex system functions, such as interrupt handlers and recovery actions.
To verify that such a system meets the design points, testing of both the hardware and firmware is required. Typically, testing is performed by running “test-cases” in a “Simulation” environment. The simulation model is based on the VHDL, which represents the current state of the hardware implementation. Test-cases, which are run in the simulation environment, include statements to initialize the simulated system and may invoke code routines to run on the simulated hardware. This code often includes firmware which has been delivered to the simulation team. Test-cases also include statements which specify the expected state of the model at the completion of execution. If the final state matches the expectation, the test-case is said to pass; otherwise it is considered a failure. Thus, the simulation environment tests the system as a whole, including both hardware and firmware execution.
A reference model, which emulates hardware behavior, has several advantages over simulation for early firmware development. The reference model environment makes use of a software model of the hardware; this model is intended to conform to the design specifications for the hardware. Since it is only a model of the intended hardware behavior (rather than a model of the actual hardware implementation), this environment can run test-cases much more quickly than a Simulation environment (on the same underlying platform).
Further, a reference model is relatively stable compared to the simulation model. Once the design has been specified, a reference model can be built without waiting for the VHDL to be implemented by logic designers. Unlike the simulation model, the reference model does not need to be updated with each release of the hardware implementation. The reference model only needs to be updated for design changes, design additions, or bug fixes for the model itself.
These advantages make a reference model the test environment of choice for firmware development. Since the model is ready earlier, firmware development can take place in parallel with VHDL development. Once the firmware has been tested in Emulation, it is delivered to the Simulation team for testing with the hardware implementation.
Frequently, simulation discovers aspects of the hardware implementation that place additional requirements on firmware in order to satisfy the design. These implementation specific peculiarities are not known to the reference model because this model is built before the hardware is implemented. Therefore, simulation is required to verify that the components will function together to meet the system design requirements.
During simulation, it can be difficult to quickly determine if a failure is caused by a defect in the test-case, the simulation model, the hardware implementation, or the firmware implementation.
In addition, the reference model may not be compatible with the Simulation test-cases. Further, since the reference model is only used for early firmware testing, it may not be entirely accurate (up to date) in the later stages of simulation. Moreover, in current solutions, comparing the simulation results to a reference model is a time-consuming, manual task, which is prone to human error. Therefore, the reference model is seldom examined during simulation.